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Abstract: 
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This study examined the feasibility of administering GED Tests using a computer based 
testing system with embedded accessibility tools and the impact on test scores and test- 
taker experience when GED Tests are transitioned from paper to computer. Nineteen test 
centers across five states successfully installed the computer based testing program, fol- 
lowed the research protocol, and transmitted testing data with minimal issues, providing 
evidence of the feasibility of administering GED Tests on computer. Two hundred and 
sixteen GED candidates participated in the research by completing two GED mathematics 
practice test forms and a survey. Participants completed the first form on paper and were 
randomly assigned to take the second form on computer or paper. The survey asked stu- 
dents to report demographic information, information about their use of computers, and 
their preference for using a computer to take tests. Regression analyses showed that par- 
ticipants were neither advantaged nor disadvantaged by taking the GED Mathematics 
Test on computer. This finding also holds true after accounting for student’s reported 
computer use and preference for taking tests on computer. 
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Background/Purpose 

GED® Tests provide a way for adults who did not finish high school 
to certify their high school-level skills and knowledge. To receive a GED 
credential, candidates must pass the GED Tests. The GED Tests cover each 
of the following five content areas: Language Arts (Reading and Language 
Arts), Writing, Mathematics, Science, and Social Studies. The American 
Council on Education, through GED Testing Service, develops the GED 
Tests and establishes minimum passing scores. Jurisdictions (including all 
U.S. states) manage all test administration procedures including ordering 
test materials, administering the tests, having the tests scored, and 
reporting results to candidates. To this end, jurisdictions lease tests from 
the GED Testing Service annually. 

As is the case with many large-scale test developers, GED® Testing 
Service has considered transitioning GED Tests from paper to com- 
puter (George-Ezzelle & Hsu, 2006). Education Week’s 2009 Technology 
Counts reported that 26 states and the District of Columbia are planning 
to administer some form of computer-based state exams in 2008-2009 
(Editors, 2009). The Graduate Management Achievement Test (GMAT) 
has been administered via computers since 1997 and the Graduate Record 
Examination (GRE) has been administered by computer since 1993. 
Computer-based test administration has the potential to save time and 
money, speed up results reporting and in the case of the GED Tests, make 
testing more accessible and make more time slots available for testing. 
Candidates generally prefer the option to test on the computer if avail- 
able; in a 2006 candidate survey, more than two-thirds of candidates 
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would have preferred taking the GED Tests on computer (George-Ezzelle 
& Hsu, 2006). In a project outside of the scope of this article, GED Testing 
Service® is piloting computer-based testing in eleven U.S. states during 
summer 2010. Results of the pilot will inform future decisions about the 
direction of computer-based testing. 

The GED test- taker population differs from the GRE, GMAT, and K-12 
student populations. An estimated 40% of students who do not finish 
high school have special needs (Lohman, Lyons & Dunham, 2008); adults 
with learning disabilities are often overrepresented among dropouts 
(Mellard 8c Patterson, 2008). Many who decide to take the GED Tests may 
require specialized support in order to access test content. Without appro- 
priate access to test items, test-takers are placed at a severe disadvantage 
in terms of demonstrating their knowledge and skills (George-Ezzelle 8t 
Skaggs, 2004). For paper-based test administrations, access to test content 
is typically provided through accommodations (George-Ezzelle 8c Skaggs, 
2004; Hsu 8c George-Ezzelle, 2008). GED Tests candidates with disabilities 
are required to apply for accommodations and receive jurisdictional GED 
Administrator approval before receiving accommodations (GED Testing 
Service, 2008). 

The types of test accommodations provided to eligible test-takers can 
vary widely and include reading aloud test items, increasing the font size or 
color contrast, providing extended time or a private space for test-taking, 
use of a calculator, allowing test-takers to produce written responses on 
computer for a paper-based test or on paper for a computer-based test, etc. 
(George-Ezzelle 8c Skaggs, 2004; Hsu 8c George-Ezzelle, 2008; Lohman, 
Lyons 8c Dunham, 2008). Of the five content areas tested by the GED, 
Lohman, Lyons, and Dunham (2008) found that mathematics was fre- 
quently most challenging for GED candidates with disabilities. Extended 
time and use of a calculator were also identified as the type of accommoda- 
tion more frequently requested. 

Like all educational tests, evidence of validity is required when eligible 
candidates use accommodations to take the GED Tests (George-Ezzelle 
8c Skaggs, 2004). Lfnfortunately, the fidelity with which test accommo- 
dations are provided to test-takers varies widely. One study that exam- 
ined the quality with which read aloud accommodations were provided 
to students identified several problems with this type of accommodation, 
including: a) the quality of the readers varies widely; b) readers occasion- 
ally mispronounce or misread words; c) readers provide intentional as well 
as unintentional hints to the correct answer; and d) test-takers are some- 
times reluctant to ask proctors to re-read parts of an item. In other words, 
students are provided with a read aloud accommodation, but the accom- 
modation itself is not delivered in a standardized or equitable manner. 
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and likely does not provide students with an appropriate opportunity 
to demonstrate their achievement (Landau, Russell, Gourgey, Erin, and 
Cowan, 2003). 

Computer-based test delivery has the potential to overcome many of 
the challenges to providing high-quality accommodations in an equitable 
manner across test centers. By providing computer-based test accommo- 
dations a testing program can ensure that accommodations such as read 
aloud and signing are delivered in a standardized manner, free of mispro- 
nunciations and unintentional clues. By allowing content presented on 
computer to be resized and/ or be presented with alternate color combina- 
tions/ contrast levels, a testing program can also avoid additional expenses 
while assuring that these access tools are available to all test takers who 
require them on the day of testing. 

The research described in this article focuses on the feasibility and 
effect of transitioning GED Tests from paper to computer, using a univer- 
sally designed computer-based test system with embedded access tools. 
More specifically, the following three research questions were the focus of 
this study: 

1. Is it feasible to administer GED Tests in a computer-based envi- 
ronment for test-takers who require accommodations and for 
those who do not require accommodations? 

2. What is the impact on test scores and test-taker experience 
when a GED Mathematics Test is moved from paper to com- 
puter? 

3. What is the impact on test scores and test-taker experience 
when a GED Mathematics Test is moved from paper to com- 
puter with embedded accessibility tools? 

Testing System 

NimbleTools, a universally designed test delivery system with 
embedded accessibility and accommodation tools, was the computer- 
based testing system used in this research on GED testing. The tools avail- 
able to candidates during GED testing were flexibly adjusted based on the 
candidate’s needs. Eor candidates who did not require a test accommoda- 
tion, NimbleTools delivered the GED Mathematics Test using a standard 
computer-based test delivery interface. For candidates who needed a given 
accommodation or set of accommodations, a test proctor settings tool was 
used to customize the tools available for each candidate. As the candidate 
took a test, s/he was able to use available tools as needed. This flexibility 
allows testing programs to customize the delivery interface to meet the 
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specific needs of each candidate and for the candidate to then use spe- 
cific tools as needed for each item on the test. In addition, NimhleTools 
collected information about the tools used for each item, which provides 
testing programs with more accurate data on the use of accommodations 
during testing. 

In its current form, NimhleTools includes 18 accessibility and accom- 
modation features. For this research, the following tools were available to 
candidates eligible for accommodations: 

• Read aloud of text with human reader 

• Magnification of text and images for candidates with moderate 
visual impairments 

• Magnification of text and images for candidates with low vision 

• Masking of test items 

• Masking of answers 

• Auditory Calming 

• Reverse contrast with selection of contrast color 

• Color overlays with selection of overlay color 

• Talking Calculator 

• Talking formula sheets 

Figure 1 (next page) displays a screen shot of a practice GED Math- 
ematics Test item in NimhleTools: 
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Figure 1 : Screen Shot of a Practice GED Mathematics Test item in NimbleTools 


1 59 Question 1 


A pointer mixes gallons of point in o large cylindrical bucket so 
thot there will be no drfference in color among indrvidiKil gallons. 


Mathematics: GED Mathematics 

OPTIONS 



If one gallon of point hos a volume of opproxinxitely 4000 cm*, 
whot Is the moximum number of whole gallons of point that con 
be poured into the bucket? 


© 

6 


14 


18 

© 

22 

© 

74 


Sound Ccmtroller 

...tv • ► 




Finish or Review 


Each test center participating in the research descrihed in this article 
was mailed a CD containing the NimhleTools application and technical 
instructions explaining how to install and launch the research test form. 
All participant item answers and accommodation tool usage information 
was transmitted from test center computers to Nimhle Assessment servers 
via the Internet. 
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Research Design 

In preparation for the GED Tests, test centers and adult education pro- 
grams frequently offer Official GED Practice Tests (OPT) to candidates in 
order to gauge their readiness to pass the GED Tests; in two of the par- 
ticipating states, Louisiana and Wyoming, OPT was required prior to 
GED testing for candidates under the age of 19 and 18, respectively (GED 
Testing Service, 2010). Eor this research, two OPT mathematics test forms 
were used to implement a repeated measures design. Eor this design, test- 
takers were randomly assigned to groups and were asked to perform hoth 
tests under the conditions listed in Table 1. To assist in random assign- 
ment, participating test centers were mailed two lists of test-taker identifi- 
cation numbers and examiners were instructed to assign id numbers from 
the first list for candidates not needing accommodations and to assign id 
numbers from the second list for candidates requiring accommodations. 

Table 1 : Test Administration Group Conditions 



Form 1 

Form 2 

Test-takers Requiring 
Accommodations 

Group 1 

Paper with 
accommodations 

Paper with 
accommodations 

Group 2 

Paper with 
accommodations 

Computer with 
accessibility tools 

Test-takers Requiring 
Accommodations 

Group 1 

Paper without 
accommodations 

Paper without 
accommodations 

Group 2 

Paper without 
accommodations 

Computer without 
accessibility tools 


The following research administration protocol was employed: 

1. Candidates were provided a short tutorial that demonstrated 
how to use the standard NimbleTools interface. 

2. Candidates were provided orientation files that demonstrated 
how to use the available accessibility tools. 

3. Candidates were provided a 7-item practice mathematics test 
to become accustomed to using NimbleTools. 

4. All candidates completed Eorm 1 on paper. 

5. Candidates randomly assigned to Group 1 completed Eorm 2 
on paper. Candidates randomly assigned to Group 2 completed 
Eorm 2 on computer. 

6. Candidates completed a paper-based 31-item survey. 
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Test centers were asked to provide standard paper-based accommoda- 
tions to participants who would normally receive accommodations for a 
GED Test for Form 1 and for Group 1 participants for Form 2. For partici- 
pants assigned to Group 2 and requiring accommodations, NimbleTools 
allowed participants to be assigned to the tools that are appropriate for 
them to use during testing while taking Form 2. 

After participants completed the paper-based test forms, examiners 
were asked to score the paper-based tests in order to provide performance 
feedback to candidates. The computer-based tests were scored automati- 
cally by Nimble Assessment research staff and reports were provided to 
test centers summarizing each participant’s performance at the item level. 
Test centers then mailed paper-based answer forms and completed sur- 
veys to the Nimble Assessment offices for analysis. 


Instruments 

Each of the two test forms used in this research contained 25 GED 
mathematics practice items, consisting of 20 multiple-choice items and 5 
short answer/grid-in items. The test forms were previously equated and 
therefore approximately equivalent in terms of difficulty. For each Form 
2 item, a read aloud script was developed. The scripts detailed how the 
test content, formulas, charts and figures were to be read aloud for can- 
didates assigned to the read aloud accommodation for Form 2. All scripts 
were reviewed and approved by the GED Test Development Director. Once 
scripts were approved, voice recordings were created. All items, voice 
recordings, and associated graphic files were placed into NimbleTools and 
were prepared for delivery. 

The 25-item OPT forms were designed to allow calculator use for the 
first 13 items, but not allow the use of a calculator for the last 12 items. 
Examiners were asked to allow participants to use the standard Casio 
fx-260 calculator for the first section of the test form (items 1-13) and 
asked participants to return the calculator after completing the first sec- 
tion. Candidates who were taking Form 2 using NimbleTools were allowed 
to use the embedded calculator (or the Casio fx-260 calculator) for section 
1. The NimbleTools embedded calculator was disabled for section 2 (items 
14-25). 

The paper-based survey consisted of three sections: demographic and 
educational background items, computer-use items, and NimbleTools 
items (taken only by Group 2 candidates). Computer-use items were asked 
to allow analyses of the potential impact of computer use and comfort 
level on the test scores of candidates assigned to take Form 2 on computer. 
Please refer to Appendix 1 (pages 27-32) to see detailed survey items. 
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Data Collection 

During a July 2009 GED Administrators conference, the GED Partner 
Outreach Director began to recruit state-level GED Administrators to par- 
ticipate in this research. In succeeding months a final roster of five par- 
ticipating states was developed. State GED Administrators then identified 
and worked with interested examiners in test centers. Participants were 
offered free administration and scoring of the two mathematics OPT test 
forms and a $5 payment per test-taker for participating in the research. 

Data were collected between October 2009 and February 2010. As dis- 
played in Table 2, 219 candidates from 19 test centers and 5 states partici- 
pated in the research. The number of participants per test center ranged 
from 3 to 38. 


Table 2: Number of Test Centers and Participants 


State 

Number of 
Centers 

Number of 
Participants 

Louisiana 

5 

61 

Missouri 

2 

20 

Nebraska 

3 

15 

Virginia 

4 

90 

Wyoming 

5 

30 

Total 

19 

216 


Table 3 (next page) summarizes demographic information about study 
participants, who were generally representative of candidates testing in 
the five states during 2009. Specifically, Table 3 indicates that 53% of 
participants were male and 47% female; the study oversampled females 
compared to 2009 gender breakout in the same five states. Black/African 
American and White represent the most common reported ethnicity at 
39% and 47% respectively; African-American candidates were oversam- 
pled compared to the 2009 proportions for the same five states. Hispanic, 
American Indian/Alaskan Native, and Asian complete the ethnicities 
reported at 9%, 3%, and 2% respectively. The most commonly reported age 
range of participants was 16-19 years, accounting for 42% of responses. 
The spread of participant age in this study is generally similar to that of 
GED test-takers at the national level, with a few exceptions; the study 
oversampled candidates 16-19 and over 30 years of age. Finally, 85% of 
participants speak only English at home, with 13% speaking English and 
at least one other language and 2% speaking only a language other than 
English at home. 
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Table 3: Demographics of Participants 


Gender (n=196) 

Percentage of Participants 

Male 

53% 

Female 

47% 

Ethnicity (n=182) 

Percentage of Participants 

American Indian/Alaskan Native 

3% 

Asian 

2% 

Hispanic 

9% 

Black/ African American 

39% 

White 

47% 

Age (n=192) 

Percentage of Participants 

16-19 

42% 

20-24 

17% 

25-29 

10% 

30-49 

23% 

50 or older 

8% 

Language Spoken at Home (n=193) 

Percentage of Participants 

English Only 

85% 

English and at least one other language 

13% 

Only language(s) other than English 

2% 


Table 4 (next page) summarizes study participants’ reported prior 
education, current employment status, and amount of time spent on GED 
preparation. Most participants reported finishing some level of high school, 
with 10% reporting that they did not attend high school and 9% reporting 
that they completed 12th grade. When asked about employment status, 
34% reported they were employed either full time or part time. Fifty one 
percent reported that they were not employed and looking for work, while 
14% reported that they were not employed and not looking for work. 
When asked to report the number of hours of GED preparation that they 
had completed through instruction, only 9% reported that they had not 
completed any test preparation through instruction, 44% reported that 
they had completed between 1 and 20 hours, 31% reported that they had 
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completed between 21 and 100 hours, and 16% reported that they had 
completed more than 100 hours of test preparation through instruction. 

Table 4: Education, Employment, and Test Preparation of Participants 


Last Year of High School Completed (n=189) 

Percentage of Participants 

Did not attend high school 

10% 

9'*^ grade 

23% 

10*'^ grade 

27% 

11*'^ grade 

31% 

12*'^ grade 

9% 

Current Employment Status (n=192) 

Percentage of Participants 

Employed full time 

19% 

Employed part time 

15% 

Not employed (looking for work) 

51% 

Not employed (retired) 

1% 

Not employed (not looking for work) 

14% 

Hours of GED Test Prep through Instruction 
(n=189) 

Percentage of Participants 

0 hours 

9% 

1 to 20 hours 

44% 

21 to 1 00 hours 

31% 

More than 100 hours 

16% 


Two survey items asked participants to report the amount of time that 
they spend using a computer on a typical day. Table 5 (next page) shows 
that 23% of participants reported spending no time using a computer at 
home and 50% reported spending no time outside of home on a computer 
each day. Responses to this survey item indicate that a sizable portion of 
the GED testing population has little or no interaction with computers on 
a daily basis. Candidates did not differ significantly in reported computer 
use, either at home or outside the home, by age, gender, or ethnicity. 
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Tables: Time Spent on Computer 


Time spent on 
computer daily 

At home 
(n=178) 

Outside home 
(n=145) 

None 

23% 

50% 

1 5 minutes or less 

11% 

14% 

15-60 minutes 

23% 

18% 

An hour or two 

21% 

10% 

Over two hours 

22% 

8% 


Participants were asked to report their computer skills according to 
four descriptions. Forty-eight percent of participants reported that they 
were either just getting started or knew a little hit about using computers, 
while 52% of participants described themselves as being experienced com- 
puter users or having advanced computer skills. 

Participants were also asked to report how long they have had a com- 
puter. Fifteen percent reported that they do not have a computer, while 
41% reported that they have had a computer for as long as they can 
remember (Table 6, next page). When asked if they had previously used 
a computer to take a standardized test, 71% of participants reported that 
they had not used a computer to take a standardized test. Again, responses 
to these items show that some portion of the GED testing population has 
little experience using computers and that the majority of candidates have 
not used a computer for testing. 
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Table 6: Computer Use of Participants 


Computer Skills (n=180) 

Percentage of Participants 

Just getting started 

10% 

Know a little bit about using computers 

38% 

Am an experienced computer user 

40% 

Have advanced computer skills 

12% 

How long have you had a computer? (n=183) 

Percentage of Participants 

Do not have one 

15% 

Less than one year 

9% 

A year or two 

16% 

Three or four years 

19% 

As long as I can remember 

41% 

Have you previously used a computer for 
standardized test? (n=182) 

Percentage of Participants 

Yes 

29% 

No 

71% 


Feasibility 

The first research question of this study focused on the feasibility of 
administering GED Tests in a computer-based environment. Evidence was 
collected by logging issues related to: 1) installation of NimbleTools; 2) 
transmission of data from test centers to Nimble Assessment servers; and 
3) logistical or technical issues in administering Form 2 on computer. Data 
were collected by logging phone calls to the help desk phone number and 
emails to the email support address. Table 7 (next page) summarizes the 
calls and emails received during research administration. 
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Table?: Summary of Support Calls 


Issue 

Number of 
Calls/Emails 

Research Design Question (assignment of ids, 
recruiting, where to send materials, etc) 

10 

System Issue Internal to Test Center 

7 

Login Question 

6 

Answer Recording Verification 

5 

Where to find answer key 

5 

Where to obtain score report 

3 

How to verify that login process works 

1 

Needed authorization to install software 

1 

Firewall Issue 

1 

Total 

41 


Table 7 shows that the most frequent issue logged related to the 
research design. Questions ranged from how to assign ids to participants, 
how to recruit candidates, how to return materials, how to receive pay- 
ment of $5 per test participant, etc. None of these issues had an impact on 
the research administration. 

The next most frequently reported issue related to seven calls from two 
test centers that had difficulty launching Nimble Tools after installation. 
Both test centers independently determined that the issues were related 
to internal network configurations. After involving IT test center staff, the 
test centers were able to resolve issues on their own and successfully have 
candidates participate in the research. 

The next five most frequently reported issues (login question, answer 
recording verification, where to find answer key, where to obtain score 
report, and how to verify that login process works) are all non-technical 
issues that typically arise in research studies involving new technology. 
Answer recording verification and verifying that the login process works 
are both issues of test administrators not being comfortable with online 
transmission of data. Examiners wanted to ensure that candidate answers 
were being saved online as candidates used Nimble Tools. In all cases the 
candidate data were correctly saved. 

Lastly, there was one instance of an examiner not having authoriza- 
tion to install NimbleTools, and there was one instance of a firewall issue 
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preventing the test center from accessing Nimble Assessment’s servers. 
Again, both issues were resolved by test center staff, and candidates were 
able to successfully participate in the research. 

All 19 test centers that chose to participate in this study were able to 
install NimbleTools, administer the online test form, and transmit answer 
information to Nimble Assessment’s servers. The above table of logged 
calls and emails shows that although a small number of technical issues 
arose in test centers, they were all resolved by internal IT staff. While this 
study is small in scale and involved self-selected test centers, it provides 
evidence of the feasibility of administering GED Tests online in test cen- 
ters. 

Impact on Test Scores 

The second research question focused on the impact on test scores and 
test-taker experience when a GED Mathematics Test is moved from paper 
to computer. Twenty of the 216 participants did not finish one of the two 
test forms. The data for these 20 candidates were not used in the analyses, 
reducing the sample size to 196. Table 8 summarizes group performance 
on the two test forms. 

Table 8: Descriptive Statistics of Form 1 and Form 2 Performance 



N 

Mean 

Std Dev 

Form 1 

Group 1 (paper) 

94 

10.99 

4.2 

Group 2 (paper) 

102 

12.64 

4.5 

Form 2 

Group 1 (paper) 

94 

10.60 

4.3 

Group 2 (computer) 

102 

11.20 

5.0 


The mean scores on Form 1 show that Group 2 performed 1.65 points 
higher on Form 1 than group 1. On Form 2 Group 2 performed 0.60 points 
higher than group 1. Overall differences in performance between groups 
on both forms was small. Examiner records indicated that while individual 
test-takers received the same length of time to take the test forms, dif- 
ferent test centers may have permitted different time lengths for their 
candidates to test. 


1-T-L-A 


Transitioning GED Tests to Computer 


Higgins, Patterson, Bozman, & Kr iz 


18 


Modal Regression 

In order to estimate the impact of transitioning GED Tests to com- 
puter, a regression model was created using Form 1 performance and 
Form 2 mode (paper or computer) to predict Form 2 performance. The 
mode variable was coded as 0 for candidates assigned to paper (group 1) 
and 1 for candidates assigned to computer (group 2). As seen in Table 9, 
the regression model accounted for 48.0% of the variance in Form 2 per- 
formance. 

Table 9: Regression Model for Mode (Paper or Computer) and Form 1 

Performance Predicting Form 2 Performance 


Variable 

Coefficient 

Standard 

Error 

Beta 

T ratio 

Significance 

Intercept 

2.381 

0.711 


3.348 

0.001 

Mode 

-0.632 

0.497 

-0.067 

-1.271 

0.205 

Form 1 

0.748 

0.056 

0.702 

13.297 

< 0.001 


Note. R Square = 0.480 


The coefficient for mode was not statistically significant (p=.205). This 
regression analysis shows that after taking into account Form 1 perfor- 
mance, there was no statistically significant difference in performance 
on Form 2 between candidates assigned to computer when compared to 
performance of candidates assigned to paper. Candidates in either group 
would be expected to perform about the same on either test form, whether 
they tested on paper or on computer. 

Computer Use and Modal Preference 
Scaled Scores 

As described in the research design section, several survey items asked 
participants to report information about their use of computers and their 
preference for using computers versus paper when taking a standardized 
test. Although the overall regression analysis reported no statistically sig- 
nificant modal effect, examining differences in performance on Form 2 
while taking into account participants’ reported computer use and pref- 
erences could show that candidates with little computer experience or 
with a preference to take tests on paper are disadvantaged by computer- 
based tests. To examine this issue, a principal component analysis was 
conducted on groups of survey items to determine the extent to which 
the items could be grouped together to form one score representing com- 
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puter use and one score representing computer preference. When items 
were found to be uni-dimensional, computer use and computer preference 
scaled scores were created for each candidate. 

Computer Use Scale 

The following six survey items were used in a principal component 
factor analysis to examine whether a single computer use scaled score 
could be created: 

1. On a typical day how much time do you spend using a computer 
at home? a) none b) 15 minutes or less c) 15 to 60 minutes d) 
an hour or two e) over two hours 

2. On a typical day how much time do you spend using a computer 
away from home? a) none b) 15 minutes or less c) 15 to 60 min- 
utes d) an hour or two e) over two hours 

3. Which statement below describes your personal use of com- 
puters? a) 1 never use a computer b) When I use a computer, 

1 am usually afraid it won’t work properly or I might break it 
c) 1 use a computer on my own, but sometimes have difficulty 
figuring out how to complete an unfamiliar task d) I use a com- 
puter with confidence and can figure out how to do just about 
anything I need to do 

4. How would you describe your computer skills? a) just getting 
started b) know a little bit about using computers c) am an 
experienced computer user d) have advanced computer skills 

5. How long have you had a computer? a) do not have one b) less 
than one year c) a year or two d) three or four years e) as long as 
I can remember 

6. I have previously used a computer for a standardized test (such 
as a state assessment, professional exam, etc), a) yes, b) no 

The principal component analysis yielded a two-factor solution, with 
factor one accounting for 42% of the variance and factor two accounting 
for an additional 19% of the variance. The first factor consisted of the first 
five items presented above. The item asking candidates if they had pre- 
viously used a computer for a standardized test was the only item that 
heavily loaded on the second factor. A principal component factor analysis 
was then run excluding the sixth survey item, yielding a one-factor solu- 
tion that accounts for 50% of the variance and had an alpha reliability of 
0.70. Factor loadings ranged from .63 - .81. This one factor solution was 
used to create a scaled score representing candidates’ computer use. 
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Computer Preference Scale 

The following two survey items were used in a principal component 
factor analysis to examine whether a single computer preference scaled 
score could be created: 

1. I would prefer using a computer to take a GED Test rather than 
using paper and pencil, a) strongly agree b) agree c) disagree d) 
strongly disagree 

2. I want to use NimbleTools to take tests in the future, a) strongly 
agree b) agree c) disagree d) strongly disagree 

The principal component analysis yielded a one-factor solution 
accounting for 83.5% of the variance, with an alpha reliability of 0.80. This 
one-factor solution was used to create a scaled score representing candi- 
dates’ computer or paper preference. 

Computer Use and Preference Regression 

A regression analysis was performed to predict Form 2 performance 
after controlling for Form 1 performance and differences in computer use 
and computer preference. Because this analysis specifically focuses on 
the impact of prior computer use and preference on computer based test 
scores, only those candidates assigned to take Form 2 on the computer 
were included. As seen in Table 10, the regression model accounted for 
45.2% of the variance in Form 2 performance. 

Table 1 0: Regression Model for Computer Use, Computer Preference and 

Form 1 Performance Predicting Form 2 Performance 


Variable 

Coefficient 

Standard 

Error 

Beta 

T ratio 

Significance 

Intercept 

3.003 

2.001 


1.500 

0.138 

Computer 

Use 

0.352 

0.458 

0.065 

0.768 

0.445 

Computer 

Preference 

0.247 

0.423 

0.051 

0.585 

0.561 

Form 1 

0.706 

0.092 

0.656 

7.703 

< 0.001 


Note. R Square = 0.452 


The coefficients for computer use and computer preference were not 
statistically significant (p=0.445 and p=0.561 respectively). This regres- 
sion analysis shows that computer use and computer preference do not 
contribute to differences in candidate test performance on Form 2 after 
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Table 11: 


adjusting for Form 1 performance. Candidates did not differ statistically 
by age, gender, or ethnicity as to whether it was easy for them to take the 
test on the computer. 

Impact on Candidate Experience 

An important factor to consider in transitioning GED Tests from paper 
to computer is the impact on candidates’ test-taking experience. All study 
participants were asked a survey question about their preference for taking 
GED Tests on computer rather than paper. Sixty-one percent of partici- 
pants agreed or strongly agreed that they would prefer to use a computer 
to take the GED Tests. Candidates did not differ significantly in preference 
by age, gender, or ethnicity. 


Computer Test Preference (Both Groups) 


Prefer using a computer to take a 
GED Test rather than paper? 

Percentage of 
Participants 

Strongly agree 

24% 

Agree 

37% 

Disagree 

28% 

Strongly disagree 

11% 

Total 

41 


Candidates who were randomly assigned to group 2 were asked addi- 
tional questions about their experience taking Form 2 on computer. When 
asked if they would prefer to use NimbleTools to take tests in the future, 
61% agreed or strongly agreed that they would prefer to use NimbleTools 
versus paper, as shown in Table 11. This percentage is identical to the per- 
centage of all participants who agreed or strongly agreed that they would 
like to use a computer to take a GED Test. One inference that can be made 
from this comparison is that taking Form 2 on computer did not influence 
candidates’ desire to take future tests on computer. 
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Computer Test Preference and Experience (Group 2 only) 


1 want to use NimbleTools to take test in future 

Percentage of Participants 

Strongly agree 

20% 

Agree 

41% 

Disagree 

29% 

Strongly disagree 

10% 

It was easy taking test on computer 

Percentage of Participants 

Strongly agree 

24% 

Agree 

51% 

Disagree 

23% 

Strongly disagree 

2% 


Candidates assigned to Group 2 were also asked to respond to the 
prompt “It was easy taking the test on computer.” Seventy-five percent of 
participants who were administered Form 2 on computer either agreed or 
strongly agreed that it was easy to take the test on computer and only 2% 
of participants strongly disagreed that it was easy taking the test on com- 
puter. It is encouraging to note that although 39% of participants would 
prefer to take GED Tests on paper, only 25% of those who were adminis- 
tered Form 2 on computer reportedly found it difficult to take the test on 
computer. 

Impact on Test Scores for Candidates using 
Accessibility Tools 

The third research question focused on the impact of transitioning 
GED Tests to the computer for candidates using accessibility or accom- 
modation tools. Of the 216 participating candidates, 22 used accessibility 
tools and finished both forms, limiting the evidence that could be collected 
to answer this research question. 
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Descriptive Statistics of Form 1 and Form 2 Performance for 
Candidates Using Accessibility Tools 



N 

Mean 

Std Dev 

Form 1 

Group 1 (paper) 

13 

9.46 

4.1 

Group 2 (paper) 

9 

10.56 

3.6 

Form 2 

Group 1 (paper) 

13 

8.69 

3.0 

Group 2 (computer) 

9 

9.56 

3.0 


The mean scores on Form 1 show that Group 2 performed 1.10 points 
higher than Group 1. Form 2 performance shows that candidates assigned 
to computer performed 0.87 points higher than candidates assigned to 
paper. The differences in scores between the two accommodated groups 
were not statistically significant across testing modes. Candidates requiring 
accommodations performed about the same on either form and in either 
mode. 

Summary of Results 

1. Although narrow in scope and relatively small in size, this 
research study provides evidence that administering GED Tests 
by computer for candidates needing accessibility tools and for 
candidates not needing accessibility tools is feasible. Nimble 
Assessment staff successfully worked with GED Testing Service 
test development, research and partner outreach staff to create 
read aloud scripts and reproduce a Mathematics Test form in 
a computer environment. Nineteen test centers in five states 
successfully installed NimbleTools, administered the research 
to candidates assigned to take Eorm 2 on computer, and trans- 
mitted data to Nimble Assessment’s servers. 

2. Candidates were generally representative of 2009 GED Tests 
candidates in the five participating states. They did not differ 
significantly in computer use, computer preference, or per- 
ception of ease of computer use by age, gender, or ethnicity. 

A regression analysis that used Form 1 and mode of admin- 
istration (paper or computer) to predict Form 2 performance 
provided evidence that candidates are neither advantaged nor 
disadvantaged by moving paper-based GED Tests to computer. 
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3. A regression analysis that used Form 1 performance, computer 
use, and computer preference variables to predict Form 2 per- 
formance provided evidence that candidates are neither advan- 
taged nor disadvantaged by prior computer use and preference 
when GED Tests are transitioned from paper to computer. 

4. The sample of participants using accessibility tools was small 
(n=22), but also shows that candidates eligible for accommo- 
dations were neither advantaged nor disadvantaged when the 
OPT was administered on computer. 

5. Although a significant portion of the research sample preferred 
to take GED Tests on paper (39%), 75% of participants that 
were assigned to take Form 2 on computer reported that it was 
easy to take the test on computer. 


Discussion 

At a qualitative level, this study yielded several valuable results. It 
demonstrated that test centers could successfully install and administer 
computer-based tests for a small number of candidates — and offered 
insights into the type of challenges they might face in getting started with 
computer-based testing and how those challenges could be resolved. 

Knowing that candidates in the study were representative of the states 
in which the study occurred is particularly useful to states with similar 
numbers of candidates, that is, those testing between 2,000 and 20,000 
GED Tests candidates annually. This evidence, while incomplete for all 
test centers in the GED testing program, is also promising for individual 
test centers in those states that serve comparable numbers of candidates. 
Future research could expand this type of computer feasibility and effect 
research to larger test centers and to additional states. 

In addition, use of Nimble Tools offered GED Tests candidates with 
disabilities who were eligible for accommodations the potential to access 
computer-based testing as equitably as candidates who did not require 
accommodations. Candidates scored about the same on both forms of 
the Mathematics OPT, even after controlling for test mode (computer or 
paper), for frequency of computer use, or for preference of computer- or 
paper-based testing. This finding, though on a small scale, is similar to 
that of George-Ezzelle & Skaggs (2004). Especially for the challenging 
Mathematics Test (Lohman, Lyons & Dunham, 2008), it may be reas- 
suring to candidates with disabilities that they would not need to be 
excluded from computer-based testing opportunities simply because they 
were eligible for accommodations. Whether they were new to computers 
or experienced computer users, whether they preferred taking the test on 
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the computer or on paper, it is critical to ensure that an avenue to com- 
puter-based testing could he open to all candidates, regardless of disabili- 
ties status. These results will add to the evidence base collected in the GED 
Testing Service computer-based testing pilot from summer 2010. 

A concern at the start of the study was whether candidates might 
differ demographically in perception of ease of use or preference for com- 
puter versus paper testing. They did not; knowing that male and female 
test-takers of different ages and ethnic backgrounds tended to share a 
preference for or against computer-based testing suggests that these pref- 
erences may be common to all candidates, at least in smaller test centers. 
Candidates’ preference for taking the test on computer, at 61%, is similar to 
George-Ezzelle & Hsu’s (2006) finding of 66%, and it is equally promising 
that candidates with and without disabilities shared similar preferences. 
The fact that three-fourths of candidates in the study found computer- 
based testing easy to use could be encouraging to future computer-based 
test-takers. Future qualitative research should investigate why approxi- 
mately 40% of test- takers preferred paper-based testing and what implicit 
barriers might be associated with that preference. 

Other suggestions for future research could include examining addi- 
tional content area tests, to see how candidates with and without dis- 
abilities would perform in reading, social studies, science, and writing. 
Candidates in both statuses could also be interviewed on their computer 
test-taking experience, to qualitatively understand what they liked and 
disliked about the experience, what challenged them, and their percep- 
tions of the test items. A larger study of candidates with disabilities who 
are eligible for accommodations would add to the evidence base. Knowing 
which accessibility tools were most often used and the effects of extended 
time or private testing space (George-Ezzelle 8t Skaggs, 2004) would also 
be beneficial. 



Transitioning GED Tests to Computer 


Higgins, Patterson, Bozman, & Krlz 

26 


References 

Editors, (2009). Technology Counts 2009, Education Week, downloaded 
on June 20, 2009 from: http://www.edweek.org/ew/toc/2009/03/26/ 
index.htm. 

GED Testing Service. (2008). Policies and procedures manual. Washington, 
DC: American Council on Education. 

GED Testing Service. (2010). 2009 GED testing program statistical report. 
Washington, DC: American Council on Education. 

George-Ezzelle, C., 8t Hsu, Y. (2006). GED candidate computer familiarity 
survey. (GED Testing Service Research Studies 2006-2). Washington, 
DC: GED Testing Service. 

George-Ezzelle, C., 8t Skaggs, G. (2004). Examining the validity of GED 
Tests scores -with scheduling and setting accommodations. (GED Testing 
Service Research Studies 2004-1). Washington, DC: GED Testing 
Service. 

Hsu, Y, 8c George-Ezzelle, C. (2008). The literacy ofU.S. adults -with 

disabilities across GED credential recipients, high school graduates, and 
non-high school graduates. (GED Testing Service Research Studies 
2008-3). Washington, DC: GED Testing Service. 

Landau, S., Russell, M., Gourgey, K., Erin, J., 8c Cowan, J. (2003). Use of 
the Talking Tactile Tablet in mathematics testing. Journal of Visual 
Impairment and Blindness, 97(2) 85-96. 

Lohman, C., Lyons, R., 8c Dunham, M., Disabilities and the GED: An 
analysis of pass rates, accommodations, and test scores. Murray, KY: 
Murray State University. 

Mellard, D., 8c Patterson, M.B. (2008). Contrasting adult literacy learners 
with and without specific learning disabilities. Remedial and Special 
Education, 29(3), 133-144. 


j-T-L-A 


Transitioning GED Tests to Computer 


Higgins, Patterson, Bozman, & Kr tz 


Appendix 1 

GED Test-taker Survey 

Instructions: 

Please place an X next to the letter of the appropriate response. This survey is 
double sided please answer items on both sides. Thank you! 

1. Gender 

o Male 
o Female 

2. Ethnicity (Select all that apply) 

o American Indian/Alaskan Native 
o Asian 

o Pacific Islander/Native Hawaiian 
o Hispanic 

o Black/ African American 
o White 

3. What language(s) do you speak at home? 

o English only 

o English and at least one other language 
o Only language(s) other than English 

4. What is your age? 

o 16-1 9 years old 
o 20-24 years old 
o 25-29 years old 
o 30-49 years old 
o 50 years or older 

5. What is the last year of high school that you completed? 

o Did not attend high school 
o Grade 
o 10^^ Grade 
o llt'^ Grade 
o 12^^^ Grade 

6. What is your current employment status? 

o Employed full time 
o Employed part time 
o Not employed (and looking for work) 
o Not employed (retired) 
o Not employed (not looking for work nor retired) 
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7. 

How many hours did you prepare for the GED Test through 
instruction? 

o 0 hours 
o 1 to 20 hours 
o 21 to 100 hours 
o More than 100 hours 

Please only answer the inext three terns if you are requesting accommodations 
on the GEDTest: 

8. 

If you request accommodations on the GEDTest, which request 
form would you fill out? 

o learning / other cognitive disabilities 
o emotional / mental health 
o attention-deficit / hyperactivity disorder 
o physical / chronic health disability 
o not sure 

9. 

Are you taking this test with extended time? 

o yes 
o no 

10 . 

Are you taking this test in a private room? 

o yes 
o no 

11. 

Are you taking this test with a supervised break? 

o yes 
o no 

Comouter Use Questions 

1. 

On a typical day how much time do you spend using a computer 
At Home? At work or away from home? 

o None o None 

o 15 minutes or less o 15 minutes or less 

o 15 to 60 minutes o 15 to 60 minutes 

o An hour or two o An hour or two 

o Over two hours o Over two hours 
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2. Which of the statements below best describes your personal use 
of computers? 

o I never use a computer. 

o When I use a computer, I am usually afraid it won't work properly or 
that I might break it. 

o I use a computer on my own, but sometimes have difficulty figuring 
out how to complete an unfamiliar task, 
o I use a computer with confidence and can figure out how to do just 
about anything I need to do. 

3. How would you best describe your computer skills? 

o Just getting started 
o Know a little bit about using computers 
o Am an experienced computer user 
o Have advanced computer skills 

4. How many computers, if any, do you have at home? 
o 0 

o 1 
o 2 

o 3 or more 

5. How long have you had a computer at home? 

o Do not have one 
o Less than one year 
o A year or two 
o Three or four years 
o As long as I can remember 

6. What type of Internet connection do you have at home? 

o Do not have an internet connection at home, 
o Use a phone line to connect to the Internet at home, 
o Use a DSL or high-speed cable to connect to the Internet at home, 
o Use a wireless connection to access the Internet at home, 
o Have an internet connection at home but I don't know much about it. 

7. I have previously used a computer for a standardized test (such as a 
state assessment, professional exam, etc.). 

o yes 
o no 
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8. I would prefer using a computer to take the GED Test rather than 
using paper and pencil. 

o Strongly Agree 
o Agree 
o Disagree 
o Strongly disagree 

Please only answer the remaining guestions if you took the second test form 
on the computer. Indicate the extent to which you agree or disagree with 
the following statements by placing an X next to the letter of the appropriate 
response.: 

1 . It was easy to take the test on a computer. 

o Strongly Agree 
o Agree 
o Disagree 
o Strongly disagree 

2. The tutorial was easy to use.. 

o Strongly Agree 
o Agree 
o Disagree 
o Strongly disagree 

3. The tutorial helped me learn how to take a test on a computer. 

o Strongly Agree 
o Agree 
o Disagree 
o Strongly disagree 

4. The tutorial helped me learn how to use the read aloud, 
magnification, and other tools. 

o Strongly Agree 
o Agree 
o Disagree 
o Strongly disagree 
o Did not use tools 
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5. The read aloud tools were easy to use. 

o Strongly Agree 
o Agree 
o Disagree 
o Strongly disagree 
o Did not use read aloud tools 

6. The read aloud tools were helpful during the test. 

o Strongly Agree 
o Agree 
o Disagree 
o Strongly disagree 
o Did not use read aloud tools 

7. The magnification tools were easy to use. 

o Strongly Agree 
o Agree 
o Disagree 
o Strongly disagree 
o Did not use magnification tools 

8. The magnification tools were helpful during the test. 

o Strongly Agree 
o Agree 
o Disagree 
o Strongly disagree 
o Did not use magnification tools 

9. The tools helped me better access test content. 

o Strongly Agree 
o Agree 
o Disagree 
o Strongly disagree 
o Did not use tools 

1 0. I want to use NimbleTools to take tests in the future. 

o Strongly Agree 
o Agree 
o Disagree 
o Strongly disagree 
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Open-Response Items 

1 . Describe any problems you had using the computer to take the test. 


2. Describe any suggestions you have for improving NimbleTools. 


Thank you for your participation in this research! 
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